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ABSTRACT 

A short term training institute conducted August 12 
to August 23, 1963, at Indiana University, Bloomington, Indiana, is 
described. Thirteen participants took part in a series of meetings, 
discussions, and activities related to evaluation problems connected 
with innovative reading programs funded under Title I and Title III 
of the Elementary and Secondary Education A.ct, The operation, 
instruction, workshops, objectives, and evaluation of the institute 
are summarized. Responses by institute participants to an extensive 
set of followup evaluation questionnaires are also included, (NH) 
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I. ADMINISTRATION OF THE INSTITUTE 

The Institute for Evaluators of Title I and Title III Reading 
Projects was a short term training institute conducted during 
August 12 to August 23, 1968. It brought thirteen participants 
to the Bloomington, Indiana campus of Indiana University for a 
series of meetings, discussions and activities concerned with the 
evaluation problems connected with the innovative reading programs 
funded under Title I and Title III, E.S.E*A«* Three members of the 
faculty from Indiana University developed and carried out the 
program and four guest speakers contributed their specific insights. 
This report briefly summarizes the operation, instruction, workshops, 
objectives and evaluation of the institute. 

SETTING UP THE INSTITUTE 

A chronological narrative of the operation of the institute 
will probably be most useful here because the very late funding of 
the institute required a deviation from the normal procedures of 
advertising for and selecting participants. 

Due to considerable budget problems in Washington, the training 
institute was not approved until July 1, 1988, and we were instructed 
not to advertise until a contract had been signed. We then sent out 
a letter to k25 school systems in a six state region asking them if 
they had anyone interested in attending such an institute from 
August 12-23, should it be funded. We received l4 applications and 
6 telephone calls. The callers wanted to inform us that such a 
training institute was needed but they could not free anyone at 
such a late notice. They wanted to come later, should we hold 
another training program next spring. 
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The ins'titute faculty then called fifteen cities to see if they 
would send participants. An additional nine names were added to the 
list of potential participants. July 1? had arrived by this time, 
and the staff requested a postponement of the institute. The 
Bureau of Research recommended, however, that the institute be held 
on the original dates using the 23 participants whose names we had. 
Letters of acceptance were then mailed to all those who had indicated 
an interest. Several called to excuse themselves because they could 
not get free afterall; one representative from Chicago started for 
the institute site but became ill enroute. Thirteen reported and 
participated. 

The only large city represented in the group of participants was 
Indianapolis, Indiana. But telephone calls from the following cities 
indicated their desire to participate in the spring of 1969 J 
Cleveland, Columbus, Chicago, San Diego, Philadelphia, Oakland, 
Buffalo. The Research Council of the Great Cities Program also 
v;anted to encourage its member cities to participate next year if 
the institute is duplicated. One applicant was on the staff of an 
evaluation center of a university. The university had a contract 
vjith a large city to conduct its evaluation of a Title I reading 
program and wanted to send the evaluator to our training institute 
to learn about the variables in reading and the instruments used to 
measure them. 

Facilities 

Participants wrre housed in a new hotel one block from the csmpus. 
One camped with his family in a nearby state parkj another chose a 
motel where his family had some playground space. Everyone reported 
that they were satisfied with their accommodations. 
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All morning sessions were held in an air-conditioned conference 
room in the student union building. Afternoon sessions were held at 
various locations, depending on the kind of activity. Most of the 
afternoon sessions were held in Fine Hall, the home of the reading 
program, ERIC/CRIER, test file and reading library. Other p.m. 
sessions were held in the computer center, the reading clinic and 
the library of the school of education. 

Cooperation of Participants 

Since all participants who applied v^ere accepted, there was no 
opportunity to screen them through the use of value criteria. (And 
perhaps this is good for we got people who desperately needed this 
kind of training.) It might have been possible to have gathered a 
group that could have contributed more from their own experience 
had they had some background in measurement and evaluation. Generally 
though, the participants worked hard in producing the assignments 
listed in another section of this report. Only one person ended 
the two weeks without submitting some work on a taxonomy of 
evaluation techniques, an outline guide for one of the workshops. 

The staff was quite pleased with the cooperation and the 
response of the participants. There was a favorable balance between 
lecture-type activities and workshop tjT-pe. 

Staff and Guest Lecturers 

Staff responsibilities v/ere divided as follows: Dr. Roger Farr 

was full time in instruction; Dr. James Laffey was 2 % time in 
instruction (information resources); Dr. Carl Smith was 50^ time 
administration and 50^ instruction. Guest lecturers gave four 
presentations on the following subjects: 
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1. International Reading dissociation and Evaluation 
"by Dr. Leo G. Fay, President of I.R.A* 

2. Unified Data Collection System for Title I 

by Dr. Richard Jaeger, Director of Title I Evaluation, 

U. S. 0. E. 

3. Reading Tests, Their Norms and Their Construction 
by Dr. Roger Lennon, Director of Test Division, 

Har court. Brace and World 

k. Classroom and Informal Evaluation of Reading 

by Dr. John Connelly, Director Reading Clinic, Fredonia 
New York University 

The combination of staff and guests enabled a variety of 
presentations and offered the participants the choice of two 
offerings on some afternoons. 

PRODUCTS OF THE INSTITUTE 

Several useful products were developed as part of the institute 
activity. Participants were encouraged to prepare an evaluation 
design for their current project and to discuss it with one of the 
staff members. Seven of the thirteen participants presented new 
designs to the staff for criticism. Each of the participants also 
reviewed in detail one of the many standardized reading tests. These 
reviews will be duplicated and made available to all the members of 
the institute. 

The participants also xforked on sections of a taxonomy of 
evaluation instruments and techniques. Each selected one part of 
the grid, a copy of which is enclosed, and located tests or prepared 
evaluation instruments. The staff had done preliminary work on the 
taxonomy and had made available check sheets, questionnaires, test 
and criteria that had been gathered as part of the preliminary 
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version of the taxonomy. The compilation of the work of the staff 
and the participants will be typed and duplicated and distributed 
to all the participants. Since it involves about 200 pages of 
material, it will require several weeks of typing, printing and 
collating. As soon as it is ready, a copy will be sent to the 
U,S,0,E, The duplicated test reviews will likewise be sent at a 
later date. 

It seems likely that this taxonomy could serve other evaluator 
training institutes and could serve as a kickoff for other materials 
that would be useful to the evaluators who attended the institute. 
(See recommendations for suggested materials for future institutes. 
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II. PROGRAM SCHEDULE AND DESCRIPTION 



Monday, August 12 

9:00-10:00 a.m. 

Introductions 
Overview of institute 

Location of facilities — test bureau, reading test files. 

Question and answer ERIC/CRIER facilities, education reading 

room. University library, etc. 

10:00-11:30 a.m. 

*'The IRA and Evaluation’* — Dr. Leo Fay, President, International 

Reading A-'SOciation 



1:00-2:00 p.m. 

Pre-test on research design, measurement and evaluation: this 

test was used as an indication of participants' strengths and 
weaknesses in the areas of measurement and evaluation. A 
reading list was keyed to areas on the exam where students 
needed extra review. 

2:00-3:00 p.m. 

Outline for evaluating reading test: this outline covered the 

mechanical and logical evaluation of a test. Included in the 
discussion were such things as the value and use of norms, the 
interpretation of reliability and validity evidence, and a 
logical evaluation of sub-tests and ? terns. 

3:00-4:00 p.m. 

Administrative details 



Tuesday, August 13 
9:00-10:00 a.m. 

Revievr of basic concepts of measurement and evaluation: this 

discussion was geared to wealmesses identified on the pre-test. 
In all cases the discussion of technical considerations of tests 
and measurement were focused on the problems of measuring 
reading behavior. 



11:00-..11:45 a.m. 
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Outline for evaluating reading tests this was a continuation 
of discussion of the test outline. Some students were better 
prepared for this discussion after the review of basic 
measurement concepts. 

l:00-.*+:00 p.m. 

Workshop on reviewing tests: ERIC/CRIEB, Institute for Child 

Study and School of Education facilities will he available. 



Wednesday, August ik 
9:00-12:00 a.m. 



Evaluation models 

CIPP| Ed. and Psych. Measurement models 5 Title I models Title III 
model: the focus of this discussion was the use of models to 

evaluation designs of local programs. Examples of various 
evaluation reports were used to point out problem areas and to 
indicate how the evaluation models could be used to improve the 
evaluation of the projects. 

1:00-4:00 p.m. 

Workshop on review of proposals and evaluation of tests. 
Participants reviewed samples of Title I and Title III proposals, 
beveral participants began the development of evaluation designs 
for their local projects. 



Thursday, August 1? 

9:00-11:00 a.m. 

Objectives: This session focused on the writing of "operational" 

objectives. This was then keyed to evaluation designs.. 

11:00-12:00 a.m. 

What is reading: (defined through measurement): The various 

±evels of reading and approaches to measurement were discussed. 
Research studies dealing with the definition of reading as 

detx- rmined by factor analysis and other techniques were also 
discussed. 

1:00-2:00 p.m. 

Ways OX collecting data: Questionnaires, check sheets, inter- 

action analysis, and other unobtrusive measur'^s were reviewed. 
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2:00-4:00 p.ra. 

Discussion of evaluation taxonomy and assignment of workshop 
activity: This session focused on a discussion of the taxonomy 

design for evaluating various aspects of a reading program. 

Friday, August 16 
9:00-9:45 a.m. 

Information resources on reading: A wide range of research 

review sources and materials were discussed and evaluated. 

10 : 00 - 11:30 a.m. 

"National Evaluation of Title I" — Dr. Richard Jaeger, U.S.O.E. 
11:45 a.m. 

Lunch with Dr. Jaeger 
1:00-4:00 p.m. 

Workshop on taxonomy: Continuation of the workshop begun on 

Thvxrsday. 



Monday, August 19 
9:00-11:00 a.m. 

ERIC and ERIC/CRIER: The goal of this session was to acquaint 

the participants with the possible uses of’ ERIC/CRIER as an aid 
•in securing information on reading progra’ms. 

11:00-12:00 a.m. 

Evaluation of teachers: Interaction analysis techniques as an 

aid in teacher evaluation were explained, demonstrated, and 
discussed. 

1:00-4:00 p.m. 

Taxonomy workshop. 



Tuesday, August 20 
9 : 00 - 10:30 a.m. 

Measuring change: The problems and techniques of determining 

whether growth has taken place were reviewed and discussed. 
Residual gain scores were explair.ed as well as other procedures 



for removing regression effects, the Hawthorne effect, and the 
effects of practice. 

10:45-12:00 a.m. 

Evaluation of materials: Methods of determining the usefulness 

and effectiveness of materials were discussed. Readability 
formulas were considered as one aspect of the evaluation of 
materials . 

1:00-3:00 p.m. 

Research design: A review of basic techniques in setting up 

control- experimental type evaluations. 

3:00-4:00 p.m. 

Taxonomy workshop. 



Wednesday, August 21 
9:00-9:45 a.m. 

Evaluating teachers: The broad problems of establishing criteria 

for determining teacher effectiveness were discussed. Specific 
procedures for conducting teacher evaluations were reviewed. 

10:00-11:45 a.m. 

“Tests and Evaluation” — Dr. Roger Lennon, Director of Test 
Division, Harcourt, Brace and VJorld — Dr. Lennon discussed 
problems of standardized tests in making evaluations. A discus- 
sion of Title I questions received by Harcourt, Brace and World 
was included. 

ll.M-5 a.m. 

Lunch with Dr. Lennon. 
l:00-4:.00 p.m. 

Taxonomy workshop. 



Thursday, August 22 
9:00-11:00 a.m. 

Measurement and types of reading programs: Measurement and 

evaluation procedures for developmental, corrective, and clinic 
type programs were discussed and various measurement devices 
wer.- demonstrated. 
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11:00-12:00 a.m. 

'’Information Retrieval”— Dr. Edward Summers, ERIC/CRIER 
1:00-4:00 p.m. 

Using canned computer programs for evaluation: Interested 

participants were taken on a tour of the Indiana University 
computer facilities. Applications of computer techniques to 
evaluation of school programs were discussed. 



Friday, August 23 
9:00-10:00 a.m. 

’’Measurement in the Classroom”— Dr. John Connelly, Director 
Reading Clinic, State University of New York at Fredonia. 
Evaluation techniques used on an informal classroom level were 
demonstrated. 

10:00-11:00 a.m. 

Problems in measuring specific reading skills: The problems 

and methods of measuring specific sub-skills of reading were 
reviewed. Instruments for measuring specific skills were 
demonstrated. 

1:00-4:00 p.m. 

Review of taxonomy activities and review of collection of 
evaluation instruroents . 

Evaluation of institute „ 



Workshops 



There were three separate workshop activities 5 institute 
participants were invited to share in all of them or to concentrate 
on one activity if they desired. 

!• _Evaluation_ of standardized tests : Participants used 

specimen sets of various reading tests to evaluate all aspects of 
the tists. The A.P.A. Guidelines were- used as a criteria base. 
Staff members werp available to discuss specific technial problems 
such as validity, reliability or norming. 



psyelppment of evaluation design : Participants reviewed 

evaluation designs from sample Title I and Title III reports which 
v;ere available. Several participants then went on to de'^’^elop 
evaluation designs for their local situations. Staff members were 
available to assist in the development of these designs. 

3« Development of taxonomy ; Prior to the institute, the staff 
members developed the concept for, and the outline of, a taxonomy 
of evaluation devices that could be used to assess various aspects 
of a reading program. The participants collected and developed a 
wide variety of instruments that could be used for evaluation 
purposes within the framework of the taxonomy. 



III. EVALUATION MW OBJECTIVES 

To evaluate the institute the directors (1) interviewed the 
participants to determine whether or not they (participants) 
thought the objectives of the institute had been attained, (2) 
obtained written responses from the participants to determine the 
effectiveness of various aspects of the institute and (3) listed 
the products developed by the institute participants. 

During the interview conducted with each participant the 
follov/ing list of objectives were presented and the participants 
were asked whether or not he (she) thought the objectives had 
been attained. 

Objectives of the Institute 

As a result of participating in this institute, the evaluators 
will be able to: 

1. Isolate and define significant variables relative to a 
reading program. 

2. Use a variety of resources for gathering information about 
reading programs on a local, state and national level. 

3. Develop criteria for evaluating significant factors in 
reading programs. 

Sv'-lect, develop and use a variety of methods for evaluating 
the variables related to a reading program. 

5. Discuss and develop models for evaluating a reading program. 

The rf-sults of the interviews with a number of the participants 
are listed below. The graph indicates hox-/ the participants 
responded to the question concerning the objectives of the institute. 
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Approximate percentages are used for ease of interpretation. 



Objective #1 


Objectives 

Attained 


Objectives Wot 
Attained 

60% 


Objective #2 


lOOjS 


0 


Objective #3 


100^ 


0 


Objective #4 


eofo 


40^ 


Objective #5 


100% 


0 


♦Overall Success 


100% 





From the personal interviews, the directors found that while the 
majority of the participants felt that the institute was highly 
successful, they (the participants) did indicate that more time 
should have been spent (1) isolating significant variables related 
to a reading program and (2) selecting and developing a variety of 
methods for evaluating the variables related to a reading program. 

To some extent, the products dsveloped by the participants will 
aid the participants in dealing with the problem areas they 
identified ■ in the interview. 

As a second aspect of the evaluation conducted by the directors 

of the institute, the participants were asked to respond in vjriting 

to the following open ended statements; (1) The institute had too 

much...., (2) The institute had too little...., (3) The faculty 

was...., (4) Specific concepts presented by the faculty were...., 

(5) I gained these specific values...., and (6) As an evaluator 

I now.... These questions were designed to provide the institute 

’•'The participants also were asked v;hether or not they thought the 
institute was an overall success. 
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directors with information concerning the content and organization 
of the institute, the effectiveness of the faculty, and knowledge 
gained by the participants. 

The following information was taken from the participants ‘ 
responses to the open ended statements. Approximately seventy 
per cent of the participants felt that the institute was very 
well balanced. (Organization and content). One illustrative 
statement concerning the institute was, "The balance was good. 

V/e each should come out with those things needed most." Regarding 
the second open ended question (the institute had too little...) 
approximately fifteen per cent of the participants felt that more 
time should have been devoted to developing the participants’ 
background knowledge in the basic concepts in the area of measure- 
ment and evaluation. Possibly this information will be valuable 
to the institute directors in developing criteria for selecting 
participants in the future. In responding to the third open ended 
question (the faculty was...) one hundred per cent of the participants 
felt the faculty was very good. Some statements made by the 
participants were 2 (1) "The balance of the faculty team was good." 

(2) "The faculty x^as grsat." (3) "The faculty was a very pleasant 
balance. I appreciated each one and the things they had to offer." 

(4) ’The faculty \ia,s vrry helpful, Imowledgable, eager to assist, 
and exuded a sense of committment to this area and the institute." 

In response to the fourth open ended question (specific concepts 
presented by the faculty.,.) the participants responded in the 
following manner. Approximately seventy per cent felt that the con- 
cepts w-'-^re presrntved very well and in proper depth. Ten per cent 
feit that the concepts were not presented in enough detail or depth. 



Approximately fiftSven per cent felt that the concepts were not 
explained thoroughly enough to see their practical application. 

In responding to the fifth open ended question (I gained these 
specific values,..) each participant mentioned a minimum of two 
specific values gained from the institute and a majority { 60 %) 
mentioned a minimum of four specific values gained from the instil 
tute. Examples of the values gained in the institute as stated by 
the participants were: (l) *'How to formulate clear meaningful 

objectives," (2) "Recognize the limits of standardized tests", 

(3) “Increased respect for observational techniques of evaluation", 

(4) "Knowledge of resources in reading," (6) Guidelines or models 
for evaluating reading programs" and (7) Pool of sample checklists 
for use in future evaluations." Examples of responses to the 
fifth open ended question which were not as specific were as 
follows. (1) "As an evaluator, I now feel more secure in carrying 
out my duties and I also realize I have much more to learn," (2) 

"As an evaluator 3 I now fed more prepared but also more cautious 
in the absolute. I fed I can be a little more realistic in the 
capacity of an evaluator." In response to the sixth open ended 
question, participants expressed their views of the institute being 
valuable for them generally because of the additional skills, 
knowledge, and tools they will be able to utilize in their positions 
as evaluators. 

The following products were produced during the institute: 

(1) evaluations of twenty standardized reading tests, and (2) a 
preliminary edition of a taxonomy for reading program evaluation. 
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SUM4ARY 



From the information obtained from the participants concerning 
the objectives, organization, content, and faculty of the institute 
for evaluators, the directors feel that it was a highly successful 
institute. In addition, the highly useful products developed in 
the institute vjill continue to assist the evaluators in better 
performing their tasks on the job. An additional folloi"-up 
questionnaire will be sent to the evaluators in November as a 
follow-up of this initial evaluation. 
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To }-‘articipants of Bvaluator’s Institute for Title I 
and Title III Reading Programs 
August 12-23, 1968 



f-n January 10, I 969 , a follow-up questionnaire was sent to each of the 
participants in the Sunmer Institute for Evaluators of Title I and Title III 
r.cadinr; iTogramr:, Pu-ust 12-23, 1968. The purpose of fiis questionnaire \jas 
to -ct a reflection of hovr the participants smi their knowledge and their 
abilit- to apply wb.ot they learned in the Institute., 

.i'.l of Iho responses that were received are recorded in a composite of 
t;ic quos-^.ionnsires attached ’-elow. Each of the items listed in the questionnaire 
coricornin ■' botl-- application and knowledge of the evaluation concepts were those 
sp ocj.j.ical.‘.p' treated in tlie ih/aluation Institute. 

:"or t*iG most part, the participants seemed to feel that they had achieved 
a fair fr.niiicrity ivitli the concepts and topics listed in this questionnaire. 

-no ..urt-or inaico’c-cd that generally they had either used or intend to use :• 
in :hcir -^recGnt situation wiiat they learned. The items listed in this ques- ■ 
tic-nnc ire are not intended to he equal in every sense. Some of these concepts 
wore prorentod in depth and others wore simply mentioned as part of the body of 
l:nowled/-r t’.at an evaluator might };avo. ihus, in the questionnaire referring 
to :;nowlcd-e proaonted during the Institute, some of the participants 
rcc.'or.ded that they x7cro not familiar with “unobtrusive measures.” “Unobtrusive 
Roacures" wore nenti oned’ very briefly as a kind of measure that is discussed 
in eval’.ai,;.on circles and tv7o or tliroe exannlevS were given. It was not 
treat’d ’In any ’:ind cf depth. 

Oj. ryor.-Gor concern is tlic fact tliat the participants did not seem to be 




vcT*' .CQnil'i'r or nou ati al3. *i7i'b!! "thG i'» ciode3. loi* evaj.uooion* 

T'.'.s T-rns 'created at some ien':th and it uas anticipated that they t-tduIcI be able 
^o roanond tr uhat content. It nay be that simply putting the initials C.I.P.P. 
ii; the oiicci-ionnairc did not offer a sufficient clue or stimulus to remind them 
trhrt i:c vrere talicing about. 






v-as -’oported in the reneral discussion of the Institute, we were 
arare -vt not onou'-h time had been spent on identifying specific areas of 
ro din" V.iat shor.ld be considered wlien evaluating a reading program. Ilany of 
bh.e cv 3 ?airtors were not reading specialists and therefore were not able to 
idr-ntify all the variables associated ^-ath the reading program. It is possible 
to "o firough. the various topics that are listed here and identify the things 
t':;t ctiould bo stressed in future evaluation institutes and those things that 
she :ld ’;e expanded or exiplained more. One of those areas that should be 
e:y'cnded is the neasr<xd.ng of specific reading sldLlls and another is different 
!:inds of measurements for different kinds of reading programs. 

•'.notijor area tlie participants felt should be expanded beyond what was 
contained in the first Institute was a discussion and a development of the 
• 'Vole for evcTu tin s.’Kjcific reading programs. This may seem lilce a request 
:^'or a cookbook a proach to 'heir o’jn reading program and that, of course, is the 
’cs.'r'' oC mrn'- of the people tjIio came to the Institute, but it also reflects 
■J;.? noed of these participants tox\t3rk through several different models and to 
'-ct aij." try to develop oval'ytoion models before being sent back to do that 
hind of work on t:\o job. 

In Gu.m‘'.-?ry, it appears that the Institute participants felt that they 
:oirinod some 'oalua1.)].o knowledge and were applying it or intended to apply 
it os situations .'.rose. There seemed to be a feeling that there was a generally 
yood balance and th.' t the topics treated se37\'-ed them well. Exceptions were 
noted in Iho previous paragraphs. 
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II. 



Evaluation Institute 
(Knowledge ) 



following list of Questxons is an attempt to deteimine how knowledge- 
able you feel concerning some of the information discussed during the 
institute* Place a check mark in the appropriate column concerning your 
knowledge of the concepts, topics, or names listed, (l) If you feel very 
Icnowledgeable and could discuss the concept, topic, or name place a check 
mark in the first column. (2) If ycu do not feel you could discuss the 
concept, topic, or name in depth but you are familiar with it, place a 
check mark in the second column. (3) If you feel that you are unable to 
either' discuss the concept, topic, or name, or are not familiar with it 
at all, place a. check mark in the last column. 



Concept, Topic, or Name 


Eot 

Familiar 


Familiar 


very 

Familiar 


1. I.R.A. 




5 


8 


2. Evaluation 




3 


10 


3. Jfcasurement 




7 


6 


it. Reliability 




9 


4 


5. Regression Effect 


3 


6 


4 


6. Hatrfchornc Effect 


1 


5 


7 


7. C.I.P.P. 


5 


6 


3 


8. Behavioral Objectives 


2 


5 


5 


9. Title I Model for Evaluation 


1 


10 


2 


10, Factor Analysis 


0 

1 


10 


1 


11, Data Collecting 


1 


8 


4 


12, Interaction Analj/sis 


3 


7 


3 


13. Unobtrusive Measure 


6 


4 


2 


lit. ERIC 




8 


5 


15. Title Three Evaluation Mel 


4 


7 


1 



Concept, Topic, or Name 


m — 

Familiar 


Familiar 


Very 

Familiar 


l6. Residual Gain Scores 


2 


9 


2 


17. Readability formulas 




9 


5 


18. Practice Effect 


4 


4 


5 


19. Control Group 




5 


7 


20. Canned Computer Proi^ram 


4 


6 


2 


21. ERIc/CRIER 




7 


6 


22. Taxonomy 




5 


8 


23. Measuring Change 


2 


6 


5 


21;. Define Reading through Measure- 
ment 


1 


5 


6 


2^. Buros 




3 


10 = 


26 . Austin, Mary, Bush, C.T, 
and Heubner, M.H. 


5 


4 


4 


27. Handbook of Research on 
Education 


2 


9 


2 



III. 



Evaluation In^stitutc 
(Practical Application) 



The following items refer to things that you may have applied. Check 
the column that best describes your use of each item since the Institute, 



1, Taxonomy of Evaluation Techniques 
for Reading 


Do not understand 
well enough to use 


Have not used it 
because it is not a 
valuable tool 


Intend to use when 
opportunity arises 


Have used once 


Have used more 
than once 

1 




1 


6 


1 


4 


?c Evaluations of Reading Tests 






3 


3 


6 


3* Evaluation Ifodcl 

(CIPP or Title I or Title III) 


1 




8 


1 


1 


kw Principles of Test Validity 
and Reliability 


2 




4 




6 


5* Model for evaluating reading 
teachers (interaction analysis) 


f 

1 




9 




2 


6, Model for evaluating reading 
materials 






6 


1 


5 


7* Outline for writing performance 
ob.-jcctivcs 


1 




9 


1 


1 


8, Outline of data collection methods 


2 




8 




2 


9. ERIC/CRIER 






7 


1 


4 


10. Principles for the measurement 
of change 


1 




6 




5 


11, V^ays of measuring specific ro''dnng 
skills 


1 




3 




8 


12, NSSE Yearbook on Reading, 1968 


1 




6 


1 


3 


13* IRA publications on evaluation 






5 




7 



r 



t 



|1 I 






ERIC 



lli. Canned Computer Programs 


6 


1 


4 


1 




1^* Procedures for developing 
local noims 


. 1 




9 


1 


1 


16, Buros* Jfcntal Pbasurement 
Yearbook 






3 


3 


6 



IV. 



EVALUATION! INSTITUTE 
(Rccommcndc'tions ) 



A* Subject matter content of the institute . Place a check in the 
appropriate column regarding your opinion of the weight given 
to the folloi-jing course content areas. 



& 

o 

•p 

o 

rt 

to 

•H 

CO 

1, Basic conccDts in moasuroment and evaluation 

9 


eo Should be ex- 
plained 


Should bo 
shortened. 


r3 

o 

T> 

o 

•p 

s 


Comments 


2. Evcoluation of rending tests 

9 


1 


2 






3, Definition of reading through measurement 

9 


3 




1 




Wajrs of collecting data 

9 


2 








3", Information resources on rooding 

10 






1 




6. ERIC and ERIC/CRIER 

11 


1 








7. Evaluation of Teachers 

8 


2 




1 




8, Measuring change 

7 


4 








9, Evaluation of materials 

7 


4 








10, Measurement in various types of 

reading programs 5 


7 








11, Problems in measuring specific ^ 

skills 


9 









CoursQ objoctivos # Plnco a chuck in the appropriate column regarding 
your estlmato o:^ the course objectives. 



.1 



Quito 

Institute particiOvants should bc' able to* Lnportant 
1.^ isolate and define the significant 
variables I'olatcd to a reading 
program. 

10 


Should be 
Ejqjanded 

3 


Should bo 
Deleted 


2. use a variety of resources for gath- 
ering information about reading ^ 

programs on a local, state, and 
national level. 


3 

1 


1 


3. Develop criteria for evaluating 

significant factors in reading ^ 

programs , 

1 

— ^ 


4 

1 




1 - ^ ^ 
select, develop and use a variety of | 

methods for evaluating the variables 

related to a reading programs* ^ 


5 


\ 


discuss and develop models for eval- 
uating reading programs, 7 


6 





6# xidditionnl objectives: (List others that you icol are very important) 



C. Activities (Assignments) Cheek the rp^^roprieto coluriin rcgnrding your opinion 
of the value of tho institute assignments; 







Should be 


Should bo 


Should be 






Continued 


Broadened 


Dropped 


a# 


Development of taxonomy 


8 


3 


1 


b. 


Evaluation of reading tests 


11 ' 


1 




c • 


Informal seminars 


8 


AL J 




d. 


Outside roacb'.ngs 


9 


3 „ J 




e. 


Formal lectures 


9 1 


2 




f. 


Examination of Title I and 


6 


6 





Title III reports 



D. Institute Porsonnel ; Chock tho appropriate column regarding the competencios 
of the institute ^taff . 



a. Knowledge in appropriate areas 

b. Interest in participants 

c. Ability to clarify confusing 
concepts 

d* Appreciation of students^ point 
of viexT 

0 , Ability and willingness to 
answer relevant questions 




E, General Suggestions for Future Institutes: 



Thank you for your assistance. 



R, Farr 
J, Laffey 
C, Smith 



1968 



